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Positive Behavioral Interventions and Supports (PBIS) is an evidence-based approach 


to addressing behavior issues in schools. A significant amount of research has been done to 


identify the critical features of PBIS. More generally, implementation science points 


to a specific sequence to ensure that PBIS is implemented with fidelity. 


This fall status report provides information on PBIS implementation fidelity for school teams who are currently in 
PBIS training sponsored by the Minnesota Department of Education (MDE). The report summarizes average 


Tiered Fidelity Inventory (TFI) scores and the types of data systems used by: 


70 school teams that are in their second year of training (Cohort 15) 
53 school teams are in their first year of training (Cohort 16) 


Tiered Fidelity Inventory 


The PBIS State Leadership Team expects that school teams in their first 
year of training (Cohort 16) will score at least 20% on their Tier 1 TFI at 
fall training. As it is their second year of training, Cohort 15 school teams 
are expected to meet the 65% implementation fidelity benchmark at fall 
training. Some key findings from the fall 2020 TFI assessments include: 


The overall average Tier 1 TFI scores for Cohort 15 decreased by 4 
percentage points since the end of the school year 2020; however, the 
overall average for Cohort 15 has increased by 31 percentage points 
since fall 2019 (Figure 1). 

The average score for all teams in Minnesota was 54% for Cohort 15, 
which was below the 65% fall benchmark for schools in year two of 
training. On average, school teams in Cohort 15 from the South region 
are implementing PBIS with fidelity according to their TFI scores, 
while school teams from the Metro and North fell just below this 
threshold. 

Cohort 16 school teams from all regions, on average, exceeded the fall 
training benchmark (20%) (Figure 2). 


The TFI measures implementation 
fidelity for each of the three tiers 
of PBIS. It is designed to be taken 
four times per year while a school 
team is in training. In Minnesota, 
schools complete a TFI at each 
PBIS training. School teams also 
complete a “walk-through” to 
interview staff and students in 
addition to completing the TFI at 
least once per school year. Schools 
in training are currently only taking 
the Tier 1 portion of the TFI 
assessment. In order to be 
implementing PBIS with fidelity, a 
school needs to score 70 percent 


or better for each of the three tiers 
of PBIS. 


1. Average Tier 1 TFI scores by region—Cohort 15 (Fall 2019 through Fall 2020) 


Metro North South Total average across all 
regions for all completed TFls 


= Fall 2019 = Winter 2019 = Spring/End of Year 2020 = Fall 2020 


Note. Total is the average across all regions for all completed TFls 


2. Average Tier 1 TFI scores by region—Cohort 16 (Fall 2020) 


29% 
21% 5 Q ae 
Metro North South Total average across 
(n=21) (n=15) (n=14) all regions (n=50) 
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Behavioral data system 


At 2020 fall training, school teams were asked to identify which ODR data system(s) they use to track data. 
Schools are expected to have an established behavior system in place by winter training in their first year. Some 
key findings include: 


=" All Metro schools in Cohort 15 report that they have a behavioral data system in place. Over half of school 
teams from the North region (53%) did not report whether or not they had a data system in place or did not 
answer this question on the fall training survey, therefore, their data system status is listed as “unknown”. 

=" Over two-thirds of Metro school teams in Cohort 16 reported they use SWIS. Most school teams from the 
North and South regions did not report whether or not they had a data system in place or did not answer this 
question on the fall training survey. 


3. Schools with behavioral data system, Cohorts 15 & 16 


Cohort 15 The presence of a behavioral data 
system allows for schools to track 


Metro office discipline referral (ODR) data 
ate) 


and use this for data-based decision- 


North making. ODR data includes whether 
(N=39) "so suedla boreslsa lees a 
possible motivations, others involved, 


The data collected in these systems 


Cohort 15 are used for improving school-wide 
Rept 30% [ism omg) am 


behavior support. 


= SWIS/SWIS+1 =lnfinite Campus =Skyward =Other data system =Unknown 


Cohort 16 
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a SWIS a Infinite Campus = Other data system = Unknown 
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Sharing key data with school staff 


At fall training, school teams were asked about how often they share implementation fidelity and outcome (ODR) 
data with school staff. School teams from Cohort 15 were also asked if they disaggregate data by gender, disability 
or special education status, and race/ethnicity. School teams from Cohort 16 were asked about their level of 
confidence in sharing these types of data with other school staff, and whether they updated PBIS artifacts on the 
current day. Key findings include: 


=" The majority of Cohort 15 teams reported sharing implementation fidelity and outcomes data at least once a 
year, though some schools from all regions indicated they have not shared or do not share this data with 
school staff (Figure 4). 

" Most of the school teams from Cohort 16 report they do not share or do not collect implementation fidelity or 
ODR data, which is to be expected since this is their first year of PBIS training. However, 37% of school 
teams from the Metro region in Cohort 16 said they share outcome (ODR) data at least once a year. 


4. Frequency of implementation and outcome data sharing with other school staff, Cohort 15 & 16 


Cohort 15 Cohort 16 

Implementation data sharing Metro North South Metro North South 
Every other month, or more often 6% 12% 0% 5% 0% 0% 
One to five times a year 75% 69% 85% 5% 20% 31% 
Do not share this data 19% 19% 15% 42% 53% 54% 
Do not collect this data 0% 0% 0% 47% 27% 15% 
Outcome data sharing 

Every other month, or more often? 13% 19% 38% 16% 0% 0% 
One to five times a year 50% 58% 31% 21% 20% 31% 
Do not share this data 38% 23% 31% 26% 53% 46% 
Do not collect this data 0% 0% 0% 37% 27% 23% 

Note. 


@ Percentages in table may not equal 100% due to rounding. Includes "Once a month or more" and "Every other month" response options 
» Includes "Three to five times a year" and "Once or twice a year" response options. 
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Schools from Cohort 16 were asked to rate their agreement with having the confidence to share implementation 
fidelity and outcome data with other school staff (Figure 5). Most schools from all three regions report they are 
confident in sharing implementation fidelity data with other school staff. While most schools from the North 
region indicated they were confident in sharing outcome data with other school staff, some school teams from the 
Metro (37%) and more teams from the South (62%) region reported lower levels of confidence with regard to 
sharing this data. 


5. Confidence level of sharing implementation fidelity and outcome data with other school staff—Cohort 16 (N=47) 


We are confident in our ability to share implementation fidelity data (i.e. SAS, TFl) with other school staff. 


Mets a 
Horn so 


= Strongly agree = Somewhat agree 
= Somewhat disagree = Strongly disagree 


We are confident in our ability to share outcome data (e.g. office discipline referrals, or ODR) with other school staff. 


as 20% 
fi om 208 
South 


= Strongly agree Somewhat agree 
= Somewhat disagree = Strongly disagree 


Schools from Cohort 16 were asked if they had updated their PBIS artifacts on the current day. A majority of 
schools in all three regions reported that they had updated their PBIS artifacts (Figure 6). 


6. PBIS artifacts updated today —-Cohort 16 (n=47) 


nae 84% 16% 


hays 80% 20% 
ra 77% 23% 


mYes uNo 
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Cohort 15 schools were asked to identify whether or not they disaggregate ODR data by race/ethnicity, gender, 
or student disability or special education status. School teams from the Metro region were more likely to indicate 
that they disaggregate outcome data by race/ethnicity, gender, or disability or special education status compared 
with the North and South regions (Figure 7). About one-third of schools in the North and South regions report 
they do not disaggregate data by gender or disability or special education status. Across all regions, between 
31-42% of school teams say they do not disaggregate data by race or ethnicity. 


7. ODR data disaggregation by region—Cohort 15 (N=55) 
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Issues to consider 


Based on the summary results described above, there are a few things the Minnesota PBIS Statewide Leadership 
Team (SLT), including the Minnesota Department of Education (MDE) and the Regional Implementation Partners 
(RIPs) should consider when planning future PBIS trainings and working with school teams. 


Most schools in training 
are have dipped below 
their TFl benchmarks—the 
SLT and the RIPs should 
continue their current 
support of schools in 
training by using data to 
customize supports for 
specific schools, especially 
in light of the COVID-19 
pandemic. 


Continue to encourage 
school teams to track 
outcome data using a 
behavioral data system. 


Encourage school teams 
to share data more often 


using creative approaches. 


Most schools from each training cohort took the TFI during the Fall training 
session. The overall average TFI scores decreased slightly from End of School 
Year 2020 training to Fall 2020 training for schools in Cohorts 15. The decrease is 
not large, and is probably expected due to the shift most schools have had to endure 
from switching to hybrid or distance learning from in-person models. The SLT and 
the RIPs should continue to think creatively about PBIS implementation fidelity and 
how to best support schools as the pandemic continues. This means using data 
(i.e. quadrant analysis, individual TFl item scores) to tailor their supports to school 
teams in order to be sure they are getting the most out of training, implementing 
PBIS in a way that is customized to their school learning model, and working to 
improve TFI scores. 


Per MDE, school teams should have a data system for tracking ODR data in place 
by Winter training of their first year. Implementing PBIS with fidelity requires collecting 
and using ODR data. Many schools did not have a data system in place as of Fall 
training. If a school team reports that they do not have a data system, RIPs should 
plan to follow up with them and perhaps provide technical assistance to getting a 
data system in place. The RIPs should also continue to emphasize collecting ODR 
data in different learning models and encourage school teams to learn more about 
how their school’s data system supports data collection during COVID-19. 


While most Cohort 15 school teams report sharing implementation fidelity at least 
once a year, some school teams in all regions report that they do not share this 
data. Between 69-89% Cohort 16 school teams from all stated they either did not 
collect or did share this data yet, however, this is not surprising given that they are 
new to PBIS. Of note, between 74-84% of Cohort 16 school teams from all regions 
indicated they strongly agreed or agreed that they are confident in their ability to 
share implementation fidelity data with other school staff. So while they may not 
collect this data or share this data, they feel confident in their ability to share it, if 
needed. The RIPs and trainers could potentially provide some technical assistance 
around sharing these types of data with other school staff, especially in the time of 
COVID-19. This could include more interactive virtual methodologies, or quick 
updates to align with the reduced time that school staff have to engage with data. 
Additionally, the RIPs and MDE could consider finding out why school teams appear 
to be less comfortable sharing outcome data. This could be done via the training 
survey, one-on-one coaching sessions with school teams, or a discussion topic for 
a regional coaches’ meeting. Moreover, Wilder is conducting a case study on SWIS 
use and reasons why school teams stop using SWIS for tracking their data. 
Perhaps we can include some question items around comfort-level and training 
around sharing ODR data as well as methods for sharing data to find out more. 
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For more information 


This summary presents highlights of the Fall 2020 Regional PBIS Trainings in Minnesota, which are sponsored by 
the Minnesota Department of Education. For more information, contact Amanda Petersen at Wilder Research, 
651-280-2741 or amanda.petersen@wilder.org. 
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